Realistically reproducing the appearance of the human face from novel viewpoints and under novel complex illumination remains a challenging problem in computer graphics due the complexity of human facial reflectance and a person's keen eye for its subtleties. The appearance of the face under given lighting conditions is the result of complex light interactions with a complex, inhomogeneous material. Realistic facial reflectance requires a model consisting of spatially-varying specular and diffuse reflectance which reproduces the effects of light scattering through multiple layers of translucent tissue.
Advances in the field of 3D scanning and reflectance measurement have enabled significant strides in the rendering of realistic faces. However, while existing methods for accurately modeling the appearance of human skin are able to achieve impressive results, it is not clear how to practically acquire the necessary parameters for these models to accurately reproduce the facial appearance of live subjects. Existing prior art acquisition techniques are either very data intensive, or they extrapolate parameters from a small exemplar skin patch to cover the whole face, or they make simplifications to the skin reflectance model.
Modeling Skin with BRDFs
In an effort to model skin appearance, some prior art techniques have utilized bi-directional reflectance distribution functions (“BRDFs”). For example, Marschner et al. [1999] use an image-based technique to obtain the aggregate BRDF of a human forehead from photographs taken under multiple lighting directions. Marschner at al. [2009] create facial renderings by modulating the diffuse component of such a BRDF with the diffuse albedo map estimated from multiple cross-polarized photographs of the face. Georghiades et al. [1999] built models of facial shape and reflectance from a small number of unknown point-source lighting directions using an enhanced version of photometric stereo [Woodham 1978]. These works assume a Lambertian reflection model, and ignore specular reflection. To account for specular reflections, Georghiades extend [Georghiades et al. 1999] to estimate a single Torrance-Sparrow specular lobe across the entire face. How-ever, they note that the lack of spatially-varying specular behavior limits the technique's ability to model the observed data, which limits the realism of the renderings. Reflectance Sharing [Zickler et al. 2006] trades spatial resolution for angular reflectance information to estimate spatially-varying BRDFs from a small number of photographs of a face. All of these methods model skin reflectance solely using BRDF models, omitting the subsurface scattering behavior of skin.
Modeling Subsurface Scattering
Modeling subsurface scattering behavior is important to create the soft, semi-translucent appearance of skin. Without subsurface scattering, renderings of skin look too harsh. Hanrahan and Krueger [1993] use a Monte-Carlo simulation to develop local reflectance models for the single and multiple scattering components of human skin and other layered tissues. Jensen et al. [2001] introduced a practical dipole model to simulate scattering behavior, and show how to infer parameters from the observation of the spread of a small white beam of light incident on a patch of skin. Donner and Jensen [2005] extend the dipole model to simulate transmission through and reflection from multiple layers, yielding a more accurate skin rendering model. More recently, Donner and Jensen [2006] presented an easily parameterized, spectrally-accurate version of the multi-layer model. These works mostly focus on practically modeling subsurface scattering for rendering. However, they do not deal with obtaining spatially-varying parameters for the dipole model or the multi-layer models. Specialized techniques, such as [Goesele et al. 2004; Tong et al. 2005; Peers et al. 2006; Wang et al. 2008], can acquire and model a wide variety of subsurface scattering materials, including skin, but are limited to planar samples only, or have acquisition times that are impractically long for human subjects.
Realistic Face Scanning
Debevec et al. [2000] use a dense sphere of incident lighting directions to record specular and sub-surface reflectance functions of a face at relatively high angular resolution. However, the model is data-intensive in both acquisition and storage. Additionally, inclusion in existing rendering systems requires significant effort. Fuchs et al. [2005] use a smaller number of photographs and lighting directions, at the cost of sacrificing continuously-varying specular reflectance. Tariq et al. [2006] use a set of approximately forty phase-shifted video projector lines to estimate per-pixel scattering parameters for faces. However, their acquisition times were as long as a minute, and they did not model the specular reflectance of skin. Weyrich et al. [2006] use a dense sphere of lighting directions and sixteen cameras to model the per-pixel specular BRDF and diffuse albedo of faces. In addition, they use a custom subsurface scattering measurement probe to obtain scattering parameters for skin. While the obtained appearance model yields impressive results, it still requires a minute to complete a full capture consisting of thousands of images.
What is desired therefore are techniques for modeling and acquisition of reflectance that address the shortcomings noted previously for the prior art.